Data communications devices, systems and networks have gained widespread use worldwide. However, they have also become more sophisticated and complex even as they become ubiquitous and crucial to the activities of enterprises and users. Further, they employ ever more complex signal waveforms in order to communicate at higher data rates and support an increased number of users and data traffic. Manufacturers, vendors and users therefore have a greater need for testing such systems.
Unfortunately, the increasing complexity of such data communication devices and systems makes them harder to test. The recent advent of wireless data communication systems has further multiplied this difficulty, as wireless devices employ both radio-frequency (RF) signals utilizing highly complex digital modulation methods such as Orthogonal Frequency Division Multiplexing (OFDM) as well as Multiple-Input Multiple-Output (MIMO) coding, in conjunction with packetized connection-oriented stateful protocols over a shared radio frequency (RF) medium. Such devices further need to support Quality of Service (QoS) sensitive applications such as voice and video, and implement advanced coding and error correction algorithms to overcome the effects of channel noise and distortion.
The requirements placed upon such wireless devices wakes them highly prone to failures due to unwanted signal errors or distortion. Therefore it is very attractive to manufacturers and users to test these devices to detect the occurrences of signal errors and distortion, to capture the signals corresponding to these errors and measure the amount of signal error and distortion, and to conduct these tests in an automated fashion. Unfortunately the complexity of the physical layer (PHY) and link layer (Medium Access Control or MAC) layer protocols makes it very difficult to identify the exact location of a signal error or distortion, as the occurrence of an error may not be detected until the entire processing has been performed and the protocol functions have been executed. Further, the extremely high signal bandwidths and consequently high data rates of such devices renders the storage of sufficient raw signal data to perform post-processing measurements very expensive and time-consuming, necessitating real-time (on-line) signal error and distortion measurements. Finally, the interactions between PRY and MAC layer protocols render the signals generated by these wireless devices unpredictable, thereby making it difficult or impossible to create automated tests for these errors.
Existing systems for testing complex wireless data communication devices fall into two broad categories: wireless traffic and protocol test systems, and wireless signal generation and analysis systems.
With reference to FIG. 1, an exemplary wireless traffic and protocol test system is represented, consisting of a Device Under Test (DUT) 100 interfaced to wireless traffic and protocol analyzer 101 by means of radio interface 103 and interfaced to wired traffic and protocol analyzer 102 by means of Ethernet interface 104. Wireless and wired analyzers 101 and 102 respectively are coordinated to send and receive test traffic by test configuration and management system 105. A set of traffic test requirements 106 may typically be drawn up describing a plurality of test cases. Analyzers 101 and 102 respectively are capable of following and comprehending communication protocols utilized by DUT 100 while performing tests, thereby making highly deterministic measurements at the packet level and above. It is possible to provide traffic test requirements 106 in computer-readable form as program data directly to test configuration and management system 105, which then performs the necessary test cases automatically without need for human intervention. However, tests performed by such an exemplary test system are limited to the packet level, protocol level, and traffic level, as such a system has no capability to define, interpret and make measurements on the actual radio or analog signals emitted by DUT 100.
FIG. 2 is an exemplary illustration of a conventional wireless traffic and protocol analyzer 101. Packetized test data traffic flows generated by test traffic generator 110 are passed to packet transmission datapath 111, which encapsulates and encodes them into a framed digital data stream suitable for baseband conversion and transmission. In an exemplary case of an IEEE 802.11 system, this digital data stream would represent PHY Layer Convergence Protocol (PLCP) frames; alternatively, in another exemplary case of Fourth Generation Long Term Evolution (4G LTE) cellular systems, this digital data stream would represent the Transport Channel (TC) data frames prior to PHY coding and rate matching. The digital data stream is then passed to transmit digital PHY logic 112 which performs the transmit digital baseband functions of coding, modulation and signal processing, and finally to transmit Digital to Analog (D/A) function 113 which converts signals from the coded digital domain to the analog domain for transmission to DUT 100. It should be understood that transmit D/A function 113 may encompass standard frequency upconversion and mixing functions if any are required.
In the receive direction, DUT 100 sends analog signals to receive Analog to Digital (A/D) function 114, which converts signals from the analog domain to the coded digital domain. It should be understood that receive A/D function 114 may encompass standard amplification, mixing, downconversion and filtering functions if any are required. The coded digital signals are passed to receive digital PHY logic 115, which performs receive digital baseband functions of signal processing, demodulation, decoding and error correction to obtain a framed digital data stream containing packets. This is in turn passed to packet reception datapath 116, which extracts and processes packets from the framed digital data stream. These extracted packets are sent to test traffic analyzer 117 to analyze the traffic generated by the DUT at the MAC layer and above.
In many cases, as in the exemplary situations of IEEE 802.11 and 4G LTE, DUT 100 may not transmit or receive test traffic unless the appropriate protocol interactions have been completed. Further, random errors and delays may cause the exact sequence of packets transmitted to and received from DUT 100 to be unpredictable; therefore, it may be necessary to implement the full ISO/OSI Layers 2 through 7 protocol functions in order to identify the correct sequences of test traffic to analyze. Finally, the exemplary wireless protocols may multiplex packet traffic flows for many different flows and end-users together into a single wireless channel, and it may only be possible to determine test traffic belonging to a specific flow or end-user by following the correct protocol. Protocol engine 118 implements the packet layer protocol functions required by the wireless communication protocol implemented by DUT 100, using the capabilities of packet transmission datapath 111 and packet reception datapath 116 to exchange packets with DUT 100. Packet flow table 119 contains traffic flow definitions and identifying fields that are used by protocol engine 118 as well as test traffic generator 110 and test traffic analyzer 117.
Turning now to FIG. 3, an exemplary conventional wireless signal analysis system is depicted, which may comprise DUT 100 including radio interface 103 and Ethernet interface 104, and connected to wireless signal analyzer 130. To cause DUT 100 to emit signals that may be analyzed by wireless signal analyzer 130, wired traffic generator 131 may be used to inject packet traffic into Ethernet interface 104 of DUT 100; these packets may then be forwarded out radio interface 103, whereupon the signals generated by DUT 100 can be analyzed in the time and frequency (spectral) domain. Typical analysis functions that may be performed on these signals include power spectral density (PSD), spectral mask compliance, error vector magnitude (EVM), and total radiated power (TRP), all of which are well known and need not be further described herein. The specific analysis functions to be performed may be set forth in signal test requirements 132. However, unlike the situation with the exemplary wireless traffic and protocol test system depicted in FIG. 1, it has heretofore proved impossible except in very specific cases to perform such analysis functions automatically on an unmodified DUT 100. Therefore, operator 133 may be required to translate signal test requirements 132 into the actual configurations of wireless signal analyzer 130 and wired traffic generator 131, and further may be required to conduct the necessary tests by manually activating wired traffic generator 131, triggering wireless signal analyzer 130, and repeating the procedure until a suitable waveform has been captured and analyzed.
An exemplary conventional wireless signal analyzer 130 is presented in FIG. 4. It may typically comprise: receive A/D function 138 that translates analog signals generated by the radio interface of DUT 100 to the digital domain; receive waveform analyzer 137 that processes the digitized signals; receive waveform memory 136 that holds the raw data generated thereby; and waveform post-processing function 135 that reads stored data from receive waveform memory 136, identifies some location within the stream of waveform data according to some property of the waveform, performs the desired signal analysis function, and displays it on a screen or other output device for the edification of operator 133. It should be understood that receive A/D function 138 may encompass standard amplification, mixing, downconversion, and filtering functions if any are required. It should further be understood that many alternate implementations of wireless signal analyzer 130 may exist, but they all share the common characteristic that the analysis function is triggered by some signal property of the waveform itself, or by some external digital or analog trigger, without any correlation to a higher layer structure that may be embodied within the signal waveform. Wireless signal analyzers therefore make the assumption that the signal stream comprises a simple predefined series of framed packets whose structure and composition are known in advance.
In the exemplary cases of IEEE 802.11 and 4G LTE, as has already been noted, DUT 100 may not transmit or receive traffic without first completing the appropriate protocol interactions. These protocol interactions may further be complex and require significant amounts of packets to be exchanged with DUT 100. In this case, wireless signal analyzer 130 may not be able to perform any analysis functions at all, because DUT 100 will not transmit any analyzable signals. Therefore, it may be necessary to utilize a wireless traffic and protocol analyzer function 101 in conjunction with wireless signal analyzer 130, as exemplified by FIG. 5. Wireless traffic and protocol analyzer 101 may interact with DUT 100 and cause the latter to send and receive packetized signals; a portion of the signals transmitted by DUT 100 is drawn off by signal tap 140 and fed to wireless signal analyzer 130, which may then proceed to perform analysis functions on these signals.
All of these exemplary instances of conventional signal analyzers and protocol analyzers suffer from the significant deficiency that they are unable to cope with the complex nature of the signals involved in modern wireless communication systems such as IEEE 802.11 and 4G LTE. Wireless communication protocols may support many clients or User Equipment (UEs) concurrently sharing a single radio channel, and may further employ contention between clients and DUT 100 to regulate access to this shared channel. Further, these wireless communication protocols may use random backoff and recovery schemes to mitigate and recover from the effects of collisions between stations, as well as error correcting codes, acknowledgements, retransmissions, windowing schemes and other complex mechanisms to handle signal errors on the wireless medium. Yet further, there may be complex interactions between various protocol layers, from the radio (PHY) to the application (Layer 7). This makes the traffic patterns actually observed on the radio channel quite unpredictable. Thus the signals emitted by wireless data communication devices may not be generally amenable to testing under the assumption of a simple, predefined sequences of packets, which is the assumption made by conventional wireless signal analyzer 130.
A further difficulty with conventional signal analysis systems is their need to capture very large volumes of raw data in order to successfully perform analysis. For instance, the exemplary system depicted in FIG. 3 may have to capture many seconds of digitized signal data in order to improve the probability of capturing at least one instance of a packet of interest. As presently known wireless data communications technologies may utilize bandwidths of many hundreds of megahertz, transferring gigabits of information per second, this causes the size of receive waveform memory 136 in exemplary wireless signal analyzer 130 to become extremely large. Also, the time required by waveform post-processing function 135 to process the data stored in receive waveform memory 136 is rendered very long, significantly impacting the usability of wireless signal analyzer 130 and reducing the rate at which tests may be performed on DUT 100.
Yet a further complexity is the presence of delay and bandwidth sensitive traffic such as voice and video in modern networks. A communications system carrying such traffic may need to reprioritize or reorder certain packets or sequences of packets in order to meet the Quality of Service (QoS) requirements of different types of traffic, and ensure that delay and bandwidth guarantees are met. The test systems for such devices and networks may therefore have to generate and analyze traffic conforming to different kinds of QoS requirements in a coordinated way, such that signal analysis results are identified with specific packets possessing predefined QoS parameters. Again, conventional test systems have difficulty in meeting such requirements.
Yet a further complexity is the need to test traffic flows associated with stateful, connection-oriented protocols. Examples of such protocols at Layer 4 of the ISO/OSI protocol hierarchy are IEEE 802.11 and 4G LTE. Another example at Layer 4 is the Transmission Control Protocol (TCP). Such stateful protocols can cause some traffic flows to stop and restart unpredictably, as the protocol state machines respond to such network events as mobility (roaming) and congestion, while other traffic flows continue unhindered. As a consequence, not only must the test systems for devices and networks implementing such stateful protocols be capable of generating and responding to such state transitions during the test, but the signal analysis on the signals generated by DUT 100 must be coordinated with the protocol functions so as to analyze the correct packets belonging to the desired traffic flows at the appropriate times. Conventional test systems have difficulty in meeting such requirements.
It should be appreciated that the above mentioned issues and requirements pertain to optical as well as wireless communications test systems. Heretofore, these issues and requirements have not received much attention in optical test systems because of the simple modulation functions used and their relative insensitivity to the characteristics of the data traffic being encoded. In addition, testing QoS functions for optical data traffic has hitherto not received much attention. However, with more complex modulation and coding schemes required to support higher data rates and the increasing need for QoS in data communication networks, it is important to account for these issues in optical test systems.
There is hence a need for improved data communication test systems and methods. A test system that can coordinate the analysis of the underlying signal waveform, with the wireless communication protocol, traffic flows, and data packets carried therein, may be desirable. Further, such a test system may preferably reduce the amount of waveform data captured during the analysis process, and may further preferably improve the efficiency of the test process and reduce the time required to conduct testing. Also, it may be desirable for such a test system to combine the protocol functions and signal analysis functions to simplify the task of causing a DUT to generate the desired signals for measurement. Finally, it may also be desirable for the test system to selectably analyze certain traffic flows with some predetermined characteristics that may preferably include properties such as QoS and TCP.